Accelerating Deep Neuroevolution on Distributed FPGAs for Reinforcement Learning Problems

Reinforcement learning, augmented by the representational power of deep neural networks, has shown promising results on high-dimensional problems, such as game playing and robotic control. However, sequential nature these problems poses a fundamental challenge for computational efficiency. Recently, alternative approaches evolutionary strategies neuroevolution demonstrated competitive with faster training time distributed CPU cores. Here we report record times (running at about 1 million frames per second) Atari 2600 games using implemented FPGAs. Combined hardware implementation console, image preprocessing network in an optimized pipeline, multiplied system level parallelism enabled acceleration. These are first application demonstration IBM Neural Computer, which is custom designed that consists 432 Xilinx FPGAs interconnected 3D mesh topology. In addition to high performance, experiments also showed improvement accuracy all compared same algorithm.